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Procedures and machinery were developed for a statewide. 5-year follow-up of 
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effectiveness analysis purposes. The system calls for specific information about the 
student to be gathered while he is in school by means of a Student Exit Form. 
Information is accumulated on magnetic computer tape, and after the student leaves 
school (graduates or drops out), he is sent a mail-out questionnaire designed to 
gather information about what the student does after leaving school. Students 
receive a questionnaire once each year for 5 years after leaving school. Exit forms 
were utilized for the first time during the 1967-68 school year and mail-out 
questionnaires were sent to 1968 graduates and to dropouts in September 1968. 
Although problems arose in procedures, data accumulation, and reports, the basic 
system has been developed, is considered functional, and is recommended for 
continuation. (Not available in hard copy due to marginal legibility of original 
document.) (CH) 
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SUMMARY 



Statement of the Problem 

, ** 

Previous examinations of the effectiveness of Utah's public vocational 
training revealed the need for accurate follow-up information on former 
vocational students. This data was needed for both secondary and post 
secondary vocational students. The problem was to develop the procedures 
and machinery for a statewide, five-year follow-up of high school and 
post high school vocational students. 

Objectives 

The objectives of the study were: 



1. To accumulate course enrollment and completion information for voca 
tional education students in Utah. 

2. To measure the progress of the vocational student in the world of 
work. 



3. To solicit comments from the former student concerning the quality 
of his vocational training. 

4. To provide the district, the school, and the instructor information 
about former students which may be used to evaluate their counseling 
and teaching effectiveness, and the value of courses taught. 

5. To provide information for cost-effectiveness analysis of vocational 
programs. 

6. To replace the forms previously required from each district and 
needed by the State to complete the Federal Follow-up Report due in 
Washington, D. C. on December 1 of each year. 

Method : 100% of all vocational students in Utah are included in the 

Follow-up program. Basic information about the student is gathered 
while he is in school by means of a Student Exit Form. The information 
is accumulated on magnetic computer tape and after the student leaves 
school (graduates or drops out) he is sent a Mailout Questionnaire. 

This form is designed to gather information about what the student does 
after leaving school; he will receive the questionnaire once each year 
for 5 years after leaving school. 

Results and Recommendations : Exit Forms were utilized for the first 

time during the 1967-68 school year and Mailout Questionnaires were 
sent to 1968 graduates and dropouts in September ' 1968. Although 
problems arose in procedures, data accumulation and reports, the basic 
system which has been developed is considered functional and is recom- 
mended for continuation. 



X. INTRODUCTION 



Intrdductory Statement 



In July 1967 a publication entitled Cost Effectivenss Study of Utah 
Vocational Programs was produced by the Utah Research Coordinating Unit 
for Vocational and Technical Education. The results of this publication 
brought to light the need of a follow-up of those students who have com- 
pleted vocational education programs. The introductory paragraph 
graphically states the need of evaluating the vocational programs. 

"A means is needed for measuring the effectiveness of our vocational 
education programs as a function of the dollars invested in these 
programs. This measure is needed as one basis for determining which 
programs need improvement or reorientation, and which programs need in- 
creasing." 

The study made some important recommendations. A recommendation was 
made to expand the former follow-on report to provide information about 
high school graduates who enter non-baccalaureate, vocational and tech- 
nical post-secondary courses. At the same time it would be salient to 
have information available on those students who go directly to work 
after high school graduation. 

Another recommendation was made to adapt the follow-on report to the 
computer so... "that plans / could / be made to gather information on 
each high school vocational graduate each year for at least five years 
to permit a better estimate of the cost-effectiveness of the vocational 
programs." 

A. Problem Statement 



A thorough study of the effectivenss of Utah's vocational education 
training courses had never been made on a statewide basis and has seldom 
been made on a district basis. Those previous follow-up programs which 
were done on an Individual district basis were often incomplete, sporadic 
or ambiguous. Host district staffs were not adequately equipped with 
the personnel, time nor money to do a thorough follow-up study. Yet, 
each district desires and needs the data which could be provided by a 
follow-up study. 

The problem was to develop the procedures and machinery for a state- 
wide, five year follow-up of high school and post-secondary vocational 
students. 

B. Objectives of the Study 

The objectives of the follow-up study of vocational students are: 
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1. To accumulate course enrollment and completion information for 
vocational education students in Utah. 

2. To measure the progress of the vocational student in the world of 
work. 

3. To solicit comments from the former student concerning the quality 
of his vocational training. 

^ * To provide the district, the school and the instructor information 
about former students which they may use to measure their counseling 
and teaching effectiveness, and the value of courses taught. 

5. To provide information for cost-effectiveness analysis of vocational 
programs. 

6 ’ 7°a * epl * ce the forms previously required from each district and 

? y 5. State to complete the Federal Follow-up Report (Form OE4045) 
due m Washington, D. C. on December 1 of each year. 



C. 



Delimitations of the Study 



f tu< * y to high school and post-secondary students 

who have begun a vocational course regardless of whether or not they 
complete a vocational program. ney 

graduates** fHf 8 * diff * Eent from other regarding high school 

? M \ *. 19 years of age many young men serve an LDS Church 

into th» “ i * S10n t0 J tM0 ° r two and a half years. In such cases, entry 
th * labor market is delayed, and evaluation of occupational develop- 
ment and success within a five year period therefore may be difficult 

3. Students who had completed a vocational class or program before the 
F01 low-up project was begun were excluded. For example, a senior 
udent who completed two years of vocational agriculture in his sopho- 
more and nunior years and took no agriculture courses the first year 
of collection (1967-68) was not initially included in the study. 

D. Review of Literature 

Patrick"! Wl 66 ’ n he - State ° f Washi ngton, under the direction of 
**!*?)' J ; Mailey * Pro J ect Researcher, initiated a follow-up system for 

the ! 8l 7 r r Para f 0ry students - This system would effectively follw 
the student after leaving school. The main purpose of the Washineton 

s udy was to determine if the former student "was employed in tUe!ccupa- 
tion for which he was trained. Using the basic Washington follow-uo P 
framework, the State of Utah is modifying that system ^meet lo^needs, 
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E. Definition of Terms 



1. COMPLETED PROGRAM: A vocational class or series of classes which 

are conducted under the standards of the State Vocational Education Plan 
and completed by the student. The criteria for secondary level program 
completion are given in Appendix I. 

2. FOLLOW-UP: A project whereby a former student is asked to complete 

a series of questionnaires to evaluate his or her educational background 
and success in the occupation for which trained; the former student is 
not '’lost" to the educators, but keeps in contact with the directors of 
''Follow-Up" via the questionnaires sent each year. 

3. FOLLOW-UP INSTRUCTION AID CODE BOOK: The book which is distributed 

to all vocational teachers and provides the instructions for completing 
the "Student Exit Form." 

4. POST -SECONDARY VOCATIONAL PROGRAM: Any program of sub-baccalaureate 

level preparation in a specific occupational field offered at a publicly 
supported technical school, junior college, four-year college or univer- 
sity in the state. 

5. STUDENT EXIT FORM: A questionnaire which is to be filled out by 

the student at the time he leaves or completes a vocational class. 

6. STUDENT MAIL-OUT QUESTIONNAIRE: The instrument which will be sent 

to the former vocational student who has graduated from or dropped out 
of school. 

7. VOCATIONAL CLASS: A class teaching a program of occupational pre- 

paration according to standards established in the State Plan of the 
Division of Vocational Education in any of the following fields: 
Agriculture, distributive education, health occupations, home economics, 
office occupations, technical occupations, and trades and industrial 
occupations . 



II . METHOD 

A. Preliminary Preparation 

In order to conduct a project of this magnitude, it was necessary to 
have the approval and support of local personnel. On November 24, 1967 
a letter was sent to Superintendents of all school districts in Utah. 
The letter explained the purpose of the program and asked for their 
approval. (See Appendix A). A total of 40 letters were mailed. Along 
with the letter were enclosed preliminary forms of the Students Exit 
Form and the Mailout Questionnaire. Fifty percent of those contacted 
responded with a letter of approval. The twenty who did not respond 



by mail were personally contacted by telephone and asked for their support 
of the project. With the exception of one school district, all were 
immediately in favor and heartily supported the idea of a follow-up 
project. It was necessary for Mr. John F. Stephens, Director of the 
Research Coordinating Unit, to meet with the Superintendent, District 
Vocational Director, and all district high school principals of the 
dissenting district to solicit their support and cooperation. As a 
result of these meetings, they too supported the ’’Follow-Up project. 

On December 5, 1967, a similar letter was sent to all post-secondary 
administrators and vocational personnel. (See Appendix B.) Again, 
we solicited their approval and support. The response was very 
favorable and 100% approval was obtained. 

On December 15, 1967, a letter explaining the ’’Follow-Up project was 
sent to all Utah high school principals and district vocational directors. 
(See Appendix C.) The letter outlined their responsibility in the 
’’Follow-Up" project. 

B. Population of the Study 

The population of this study includes 1007* of the high school students 
in Utah who enroll in a vocational class, as well as all post-secondary 
vocational-technical students. Inclusion of all students who enroll in 
vocational classes and programs permits a comparison of those students 
who complete a program with those who leave the program before completion. 
A 100% follow-up was chosen instead of a sampling method because a 
major objective of the program was to supply information about its own 
students to each high school (and post-secondary school) in the state. 

Many of Utah's high schools are extremely small in size and their voca- 
tional student populations were considered too small to survey on a 
sampling basis. 

C. Student Exit Form (See Appendix D) 

The Student Exit Form is to be completed by each high school or post- 
secondary school student who exits for any reason from a vocational 
education class, regardless of whether the course has been completed. 

The data obtained from this form include: Marne, address, social security 

number, an alternate address, district and school codes, course name 
and code, exit date, current status of the student (dropping out, 
remaining in school, etc.), birthdate, and whether the student completed 
the course.^ The form is designed for ease and speed of completion. 



1-The teacher's name and social security number were also requested 
on the form illustrated in Appendix D; it was later decided that this 
information was not necessary. 
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D. Enter Information on Tape 

After the Exit Forms are accumulated, the information is keypunched and 
then entered on magnetic computer tape where each student is identified 
by a social security number. A separate record is established for each 
student within occupational fields. For example, if student A has taken 
two agriculture courses and two courses in office occupations, a record 
for that student would be set up in the agriculture area and in the office 
occupations area. The record in «, jch area is cumulative: the student 

described above would be listed as having taken two courses in each field, 
rather than being listed once for each course. In addition, if this 
student took an additional agriculture course the following year, a new 
record would not be established, but his total courses in agriculture 
would simply be increased to three. 

It was deemed desirable to develop a method of identifying special 
groups of students on whom Follow-up information might be requested, 
without the necessity of knowing the social security numbers of such 
persons. For example, the evaluation of several innovative programs 
involves following the progress of students enrolled in them. To simplify 
the processing of this information, a special coding system was established. 
A three-digit code is assigned to each special group and entered on the 
records maintained for students in these groups. Whenever it is desired 
to retrieve information on one of these groups, this can be accomplished 

by sorting out ail records with the particular special code assigned to 
that group. 

E. Newsletters 

Communication was maintained between the state-level direction of Project 
Follow-up and local school personnel who were responsible for supplying 

the basic data by means of newsletters which were written and mailed as 
needed . 

F. Mai lout Questionnaire (See Appendix E ) 

During September of each year for five consecutive years after he graduates 
or otherwise leaves school, the vocational student is sent a mail out 
questionnaire. The questionnaire is pre-printed on a standard keypunch 
card and sent to the student in a "window** envelope. The card format is 
designed so the name and address, printed by the computer, will show 
through the window of the envelope. A pre -stamped, pre-addressed 
envelope is enclosed for the respondent to use. After 10 days, a re- 
minder letter is sent to all non- respondents. After another 10 days, 
the non-respondents receive another questionnaire j in cases where the 
first questionnaire was returned because of incorrect address, the second 
questionnaire is sent to the alternate address supplied on the Student 
Exit Form. All identification of non- respondents and addressing of 
reminder notices and second questionnaires is done on the computer so that 
clerical work is minimized. The keypunch card was selected for the 
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mailout questionnaire because identifying information could be key- 
punched into the card to enable computer-recording of non-respondents 
as described above, as well as rapid sorting of the cards. 

G. Non-Response on Mailout Questionnaire 

Those former students who do not respond to the mailout questionnaire 
after the three efforts outlined in the previous section are removed 
from the master tape. The intention is that students who have been 
dropped due to non-response the first year will not be contacted in the 
following year's mailout questionnaire survey, on the premise that 
response from this group would be minimal. However, consideration is 
being given to including a sample of this group in the Follow-up survey 
during the second year in order to determine if the premise is correct. 

H. Reports Supplied 

Data for the Federal Follow-up Report (Form OE 4045) is compiled from 
returns on the Mailout Questionnaire. In addition, analyses are pre- 
pared for individual schools and districts in the state. The information 
supplied in these local reports is shown in Appendix F. 

These reports are assembled by the computer, based on the data collected 
from the Exit Forms and Mailout Questionnaires. 

I. Special Non-vocational Follow-up 

One fact which became increasingly obvious as Project Follow-up developed 
was that there was little information available on non-vocational 
students against which to compare the data accumulated on vocational 
students. It was decided, therefore, that a special follow-up study 
would be undertaken which would include all non-vocational students in 
the 1968 graduating class in four high schools in the state, East (Salt 
Lake District), Payson (Nebo District), Richfield (Sevier District), and 
Parowan (Iron District). 

These students will be sent a Follow-up survey questionnaire every year 
for five years following graduation. The sequence and procedures followed 
are the same as those for the regular vocational follow-up. A new 
questionnaire was developed for this group (see Appendix G) because the 
main questionnaire had been specifically designed for vocational students. 

J. Cost Effectiveness Analysis 

One of the objectives of Project Follow-up as previously enumerated 
was to provide data for cost-effectiveness analysis of vocational-technical 
programs. A previous cost-effectiveness study referred to in the intro- 
ductory section of this report pointed to the need for accurate follow-up, 
information on vocational students in order to determine the effectiveness 
of vocational programs as a function of the dollars invested in them. 



"Effectiveness’ 1 is defined in the present report in terms of "successful" 
former students, who in turn may be enumerated as those who: 

1. enter full time employment in the field for which they were trained 
or a related field; 

2. enter post-secondary educational programs in the same or related field. 

In addition, it may be assumed that some portion of the former students 
now serving in the military forces would otherwise be employed in the 
same or a related field or would be pursuing studies in the same or a 
related field. To account for these people, one quarter of the total 
number of former students in the military are added to the number defined 
as successful. The "one quarter" was arbitrarily selected, and has not 
been validated. 

Data on the dollars invested in vocational programs are accumulated from 
reports filed by the school districts with the Division of Vocational 
Education. 

Due to the difficulty in accurately amortizing costs of facilities and 
equipment used in the vocational programs, these costs were eliminated 
from consideration. Salaries of vocational personnel, plus their in- 
cidental expenses such as travel, supplies, etc., were used as the sole 
basis of dollar costs. These costs included federal, state, and local 
monies, and basically represent "0 and M" costs. 

A method of handling capital expenditures as well as administrative 
cost 3 should be developed so that the cost effectiveness figures will 
better reflect the different inputs of schools and districts. In addition, 
at present, cost figures are submitted from local school districts and 
post-secondary schools only; individual high schools do not send cost 
data to the state Vocational Education Division. Submission of this 
information from the high schools should be initiated. 



III. RESULTS 

A. Return of Student Exit Forms 

Student Exit Forms were received through July 1968 for the first year's 
records. A total of 22,871 separate documents were processed, which 
constituted 19,611 separate records. The difference between these figures, 
3,260, is attributable to students who had more than one course in a 
particular vocational field and, therefore, constituted only one record 
in that field. An unduplicated count of students, counting each student 
only once regardless of how many vocational classes he took and whether 
or not they were in one or more vocational fields, revealed that 18,850 
individuals were enrolled in vocational-technical classes in Utah 




during 1967-68 school year. This includes both high school and post- 
high school students 

In advance of the receipt of all Exit Forms it had been expected that the 
total number of forms processed would, in fact, be only a portion of the 
total number which should have been tabulated. This is attributable to 
such factors as entire classes or schools failing to submit Exit Forms; 
students absent on the day of Exit Form completion; unusable forms due 
to lack of Social Security numbers, or other vital information; and 
similar circumstances. To handle this possibility and to provide a way 
of correcting for it, a check count procedure was developed. Enrollment 
figures were obtained from 15 schools for all vocational classes for 
which Exit Forms should have been submitted. When the count of students 
was available from the Exit Form tape, the total obtained from the check 
count was compared with the recorded total from the same schools. An 
adjustment factor was then calculated to correct for the difference between 
the two figures. See Appendix H for the data. All reports based on the 
Follow-up data were corrected using this factor, except where noted 
otherwise. 

Returns of Student Exit Forms were particularly poor for schools at the 
post-secondary level. This is attributable to the fact that staff efforts 
were unintentionally concentrated on the high schools, and to the mis- 
understanding of definitions and procedures among the post-secondary 
personnel. This misunderstanding resulted from the fact that differences 
between high school and post -secondary programs were not adequately 
accounted for in the instructions furnished to the schools. Because of 
this poor return, the check count procedure described above was not used, 
as it was felt that Exit Forms submissions bore little resemblance to 
actual enrollments and completions. 

Incomplete Forms: Social Security Numbers 

Considerable numbers of Exit Forms were submitted with some information 
lacking, which necessitated returning forms to the schools to be completed. 
This problem was particularly acute in the case of Social Security numbers. 
During the first year of the program extensive efforts were necessary to 
gather social security numbers, including repeated contacts with school 
personnel, letters sent directly to students, telephone communications, 
etc. Because of the time and money required for this operation, it 
was decided that in future years students for whom numbers could not be 
obtained after reasonable efforts would receive a computer-assigned 
identification number derived by an algorithm from such basic data as 
last name, birth date, etc.' 

B. Return of Mailout Q uestionnaires 

$824 questionnaires were sent to former students in September 1968 and 
of these, 5,625 questionnaires, or 64%, were returned. A record was 
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maintained of the rate of return of the forms, and this was examined to 
determine if the reminder notice and second questionnaire mailout were 
effective in increasing returns. In both cases a noticeable increase 
occurred In the return of questionnaires for several days after the 
mailing. As might be expected, this was more pronounced in the case of 
the second questionnaire than of the reminder notice, but in view of the 
fact that the former was also more expensive, the procedure followed is 
considered satisfactory. The same procedure was used in following up 
non^vocational students in four high schools. A return of 380 of the 
495 questionnaires sent was obtained or 77%. A question arose concerning 
the possible bias in the data introduced by non-response of 36% of the 
survey group (3,199 persons). In the first year of the Follow-up a 
sample of the non-responders was surveyed by telephone in order to 
establish whether the response pattern of this group corresponded to that 
of the respondents. No major discrepancy was uncovered. Further in- 
vestigation should be done of this problem and alternate ways of handling 
non-response error should be considered. 

C, Federal Follow-up Report 
Secondary School Level 

Data for the Federal Follow-up report (Form OE-4045) was accumulated from 
the Student Exit Forms and the Mailout Questionnaires. Considerable 
change was made in the data supplied by the computer before it was 
officially submitted to the U.S. Office of Education. These changes can 
be described as follows: 

1. The non-responders were distributed proportionally among the various 
categories to reflect the distribution presented by the responders. It 
was considered legitimate to assume that the non-responders actually 
resembled the response group in view of the telephone survey of those 
from whom a reply questionnaire was not received (see above). 

2. Differences existed between the total number of completions reported 
to the Vocational Education Division in the Spring of 1968 and submitted 
to USOE on form 0E-4048, and the number computed on the basis of the E x .it 
Form submissions, even when those figures were adjusted using the adjust- 
ment factor previously described (see p. 10). The final copy of the 
Follow-up report was made to conform to the totals indicated on Form 
0E-4048; it is hoped that with some further experience the Follow-up 
procedure will supply data of sufficient validity and reliability that 
such changes will not be necessary. 

Post-Secondarv Level 

Because of the very incomplete data supplied by the post-secondary schools, 
the 1968 Report was based entirely upon data supplied by the Division of 
Vocational Education. 
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D. Local Reports 



Follow-up reports were computed for each high school, district and post- 
secondary school in the state. Information concerning the former voca- 
tional students from each school and district was supplied which indicated 
how many were working and at what types of jobs; how many were attending 
school and in what kind of programs; the number in the Armed Forces; 
average salaries of those employed; and the responders' opinions of the 
value of their training. The format and explanations for this report 
are shown in Appendix F. 

The following problems and information arose when the Local Reports were 
first issued in 1969: 

1. The adjustment factor described on p. 10 was applied to all questions 
on this report except #14 and #15, both of which dealt with dropouts, 

#19, average salary, and #20, % of response. 

#14 and #15 were not adjusted because there had not been a special check 
count to establish a factor to use with dropouts. Such information will 
be gathered and a factor established to use in future reports. 

2. The data on average salary was not very accurate on the report issued 
during the first year of Follow-up because information supplied on the 
Mai lout questionnaires was inconsistent. The question regarding salary 
requested monthly salary but replies included hourly and weekly as well 
as monthly figures. This question will be revised to provide salary 
range brackets and ask responders to check the appropriate response, 

3. The format for the local report did not include a separate count of 
those who did not respond to the mailout questionnaire, nor were they 
included as "status unknown"; they were however included in the total 
number completing program requirements. Consequently, the totals of 
former students attending school, employed or looking for work, in the 
armed forces, not in the labor force for other reasons, and status 
unknown did not equal the total number completing program requirements. 

A separate counter of non-responders will be added to future reports to 
overcome this problem, 

4. U of response - This item was incorrectly defined on the original 
computer program. It was figured as follows: 

# of responders/disciplin e (unadjusted ) 

# of completions in all disciplines/school (unadjusted) 

The correct equation is: 

# of re s ponders/discipline (unadjusted) 

# of comp let ions /disc ip line (adjusted) 
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5. Cost Effectiveness: A crude measure of the cost effectiveness of the 

various state-wide programs in each vocational area was computed by 
dividing the total amount of 0 & M dollars spent in the programs in FY 
1963 in that field by the number of '‘successful" students. A successful 
student was defined to be one who completed a vocational program and 
graduated from high school in June 1963, obtained employment in the same 
or related occupation, or attended post-secondary schooling in a related 
area, plus 25% of those who completed the program and entered the military 
service. The costs per successful student thus computed are listed below: 



Agriculture 


- $2,961 


Distributive 


- $1,592 


Health 


- $ 230 


Home Economics 


- $3,188 


(Gainful) 




Bus iness 


- $ 978 


Trade and Industrial 


- $1,599 



It should be pointed out that the number of students involved in health 
and home economics (gainful) are relatively small and the indices derived 
are of doubtful validity since a variation of only a few students made 
a large variation in the computed monetary value. It should also be 
noted that the money spent in each field was distributed in some manner 
among grades ten through twelve whereas the successful students were only 
those completing twelth grade. Therefore it could be argued that the 
figures above if divided by three would give more logical results. 
However, whether or not divided, the numbers above provide a good basis 
for comparing the costs to provide vocational education to a "successful" 
student. Only special vocational 0 & M expenditures were considered so 
that facility and equipment costs and regular DU costs would have to be 
added to derive a total figure. 



E. Problems Encountered with Definitions 

i 



Each step of the Follow-up project involved problems of defining terms 
operationally so that they would be understood by those supplying the in- 
formation, translatable into data processing procedures, and/or consistent 
with officially designated standards, both state and national. Such 
definitions generally were established after consultation with the 
specialists and administrators in the Division of Vocational Education. 

i A partial listing of some of the terms which constituted definitional 

\ problems is provided in Appendix J, for reference by persons who might 

i be in a position to need such definitions. 

Y 
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IV. RECOMMENDATIONS 



1. Project Follow-up should be continued 



2. The collection of follow-up information on vocational students should 
become a routine phase of data accumulation through the Division of 
Auxiliary Services with aid supplied by the Research Coordinating Unit 
when needed. 

3. A sample of the previous non-respondees to the mailout questionnaire 
should be surveyed to establish whether their response pattern is similar 
to those who did respond. 

4. The information being gathered by the Follow-up forms (Exit Form 

and mailout questionnaire) should be reviewed periodically by the RCU and 
the Vocational Education Division to determine if additions or changes 
are required. 

5. Continuing efforts should be made to refine the cost-effectiveness 
analysis. These efforts should include acquisition of cost data from 
individual high schools, and development of procedures for handling capital 
expenditures and administrative costs. 

6. Further consideration should be given to surveying each year's 
graduates at the end of a ten-year period in order to overcome possible 
delimitation discussed on page 2, section C, #2, 
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UTAH STATE BOARD OF EDUCATION 

1400 UNIVERSITY CLUB BUILDING • 136 EAST SOUTH TEMPLE 

SALT LAKE CITY, UTAH 84111 

APPENDIX A 




Office of the 

STATE SUPERINTENDENT 
OF PUBLIC INSTRUCTION 

T. H. BELL 

Superintendent 



The State of Utah recognizes the importance of modern and flexible vo- 
cational and technical programs in the public and private schools of 
the state. In fact, there are numerous examples of innovation in our 
school programs which attempt to provide more realistic vocational 

[ learning experiences for our students. However, a thorough study of 
the true effectiveness of these training courses has never been made 
on a state-wide basis and has seldom been made on a district basis. 

I Beginning in 1966, the State of Washington began a program which would 

I effectively follow students who have had vocational preparatory training. 

I The Research Coordinating Unit of the Utah State Board of Education is 

[ adapting and modifying Washington's follow-up system to fit the needs of 

I Utah. The purpose of this experiment is to follow vocational students 

I for five years after high school graduation and to provide each district 

| and/or school with data which can be used to evaluate, improve and change 

its vocational and technical programs. At the same time the state will 
have more consistent data to measure the progress of the state as a 
whole, as well as providing information for federal reports. Electronic 
data processing techniques, together with direct communications between 
"Computer Central" and former students, will be used in the program. 

The social security number of each vocational student will be the chief 
means by which the computer will be able to identify and trace former 
students despite changes of names, etc, 

? 

The follow-up technique to be used involves several basic steps: 

1. Having each student who reaches the 10th grade secure a social se- 
curity number no later than the fall semester. 

[ 

I 2. Having each 10th, 11th, and 12th grade student who is enrolled in a 

; vocational class, assisted by his teacher, complete an exit interview 

form which is forwarded to: 

\ "Follow-up" 

1400 University Club Building 
Salt Lake City, Utah 84111 

I 

[ For students enrolled in vocational and technical courses, these exit 

\ interview forms must be completed and forwarded immediately prior to 

the student's departure from the course -• regardless of the reason for 
| leaving (successfully completing, moving, dropping out, marriage, etc.). 

[ A tentative copy of the exit interview is enclosed. This form should 

be completed for each vocational and technical course taken by the 
I student. 
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o; t obr^“ iC ! h tin f With every former vocational student each year in 
of the tf^J tlV ?, yearS foUowin 8 departure from high school. A copy 

^^ B ssir-“ ra 2. , iss“ questionnaire which wii1 •« 

gest ions" c h ° £ 3 ° £ a ^no„ledgement. Any comments /sug- 

of f-hp n l criticisms will be appreciated. We will keep you informed 
f the progress of this program. We will be most happy to answer anv 
questions you may have concerning this follow-up program. Such inquiries 

ordinatInfunit me i3oS ***’-*»»*»' Sector, Research Co- 

ro mating Unit, 1300 University Club Building, Salt Lake City, Utah. 

Dossihlp tte Th t ^ ng t0 deve }°P the Exit Interview Form as uncomplicated as 

tion «-,•«-£ lt 1S n0t ° Ur intention t0 burden the teachers and administra- 
tion with a copious amount of paper work, aamimstra- 

bute t d e ro e aU f nreh e J U r ti . Cie ? ° f the eXit int «vie„ forms will be distri- 
all . Ut f h hi 8 h schools. We will contact the individual hiph 
school principals and vocational teachers to instruct them in the frn 
cedures which they are to follow. Also, a "code boo^" win he ? \ 

i^vLrr 1 ^ 1 h CeaCh6r f ° r USe in the 0k parts U of e th P e r °I’it d 

interview form which require a school number or course number. 

“d1uggestio„r r C0 ° Perati0n - We are look -8 forward to your comments 
Sincerely, 



John A. Cox, Research Assistant 

Enclosures : Exit Interview Form 

Student Mail-out Questionnaire 
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APPENDIX B 



The State of Utah recognizes the importance of modern and flexible 
vocational and technical programs in the public and private schools 
in the state. In fact, there are numerous examples of innovation 
in our school programs which attempt to provide more realistic 
vocational learning experiences for our students. However, a tho- 
rough study of the true effectiveness of these training courses has 
never been made on a state-wide basis and has seldom been made on 
a post-secondary vocational- technical basis. 



Beginning in 1966, the State of Washington began a program which would 
effectively follow students who have had vocational preparatory train- 
ing. The Research Coordinating Unit of the Utah State Board of Educa~ 
tion is adapting and modifying Washington's follow-up system to fit 
the needs of Utah. The purpose of this experiment is to follow voca- 
tional students for five years after high school graduation and to 
provide each district and/or school with data which can be used to 
evaluate, improve, and change its vocational and technical programs. 

The procured data will provide the state with more consistent data to 
measure the progress of the state as a whole, as well as providing 
information for federal reports. Electronic data processing techni- 
ques, together with direct communications between "Computer Central" 
and former students, will be used in the program. The social security 
number of each vocational student will be the chief means by which 
the computer will be able to identify and trace former students des- 
pite changes of names, etc. 

The program has two phases. First, the student is asked to fill out 
an exit interview form (with the instructor's help, if necessary) 
shortly before the student leaves a course. The form is to be com- 
pleted regardless of the student's reason for leaving the course. 

These completed forms are to be sent to: 



A tentative copy of the Exit Interview Form is enclosed. 

The second phase consists of a student questionnaire which will be 
sent to the former student for five consecutive years. The data 
from these questionnaires will be analyzed and the information will 
be returned to the instructors in the form of an Instructor's Record, 
The results of the exit interview forms and the student questionnaires 
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"Follow-up" 

1400 University Club Building 

136 East South Temple 

Salt Lake City, Utah 84111 



will be made available to the school involved. The tentative Student 
Mail-out Questionnaire j.i enclosed. 

We are attempting to make the Exit Interview Form as uncomplicated as 
possible. It is not our intention to burden the vocational teachers 
and administration with a copious amount of paper work. 

We earnestly solicit your cooperation in this project. Before embark- 
ing on this program, we are asking for a commitment of approval from 
your office in the form of a letter of acknowledgement. We are in no 
way asking for monetary assistance. We will be most happy to answer 
any questions you may have concerning this follow-up program. Mr. John 
F. Stephens, Director, Research Coordinating Unit or John A. Cox, 
Research Assistant, will be available, at your request, to brief the 
vocational-technical faculty about this program. Any comments, sugges- 
tions, or criticisms will be appreciated. 

In the near future quantifies of exit interview forms will be distri- 
buted to all Utah high schools and post-secondary vocational schools. 

We will contact the department heads and vocational teachers to instruct 
them in the procedures which they are to follow. Also, a "code book" 
will be provided to each vocational teacher for use in completing the 
parts of the exit interview form which require a school number or 
course number. 

Thank you for your cooperation. We are looking forward to your comments 
and suggestions. 

Sincerely yours, 



John A. Cox 

Research Assistant JAC:mh 

Utah Research Coordinating Unit Enclosures 2 
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APPENDIX C 



A state-wide experimental program will begin in January in the 
school districts of Utah. The program will attempt to follow the 
graduates and drop-outs from the secondary schools for five conse- 
cutive years after they leave school. The object is to measure the 
progress of these students by examination of their post-secondary 
employment accomplishments. It will also solicit and categorize 
their suggestions as to the benefits and shortcomings of the educa- 
tional programs in which they participated while in high school. 



In the beginning, only the students in the vocational classes will 
be involved; then, after a year or two of working the "bugs" out, if 
the program proves to be satisfactory and useful, the program will be 
expanded to include all students in our high schools. 



Basically the program has two phases, each of equal importance. First, 
each student in a vocational class will be asked to complete (with the 
instructor's help, if necessary) an exit interview form shortly before 
he leaves the course. This exit interview form must be completed 
regardless of the student's reason for leaving (e.g. marriage, dropping 
out, illness, etc.). Once these forms are completed, they will be 
forwarded by the secondary schools to: 



"Follow-up" 

1400 University Club Building 

136 East South Temple 

Salt Lake City, Utah 84111 



A tentative copy of the Exit Interview Form is enclosed as Enclosure 1. 



Once the Exit Interview Forms are received in Salt Lake City, the data 
will be processed by computer and entered on tape. An Instructor's 
Record will be printed out and returned in multiple copies to the 
participating school. The instructor will then have in his possession 
a record of his former vocational students. A tentative copy of the 
format of the Instructor's Record is enclosed as Enclosure 2. 



This essentially completes the first phase of the program. This process 
will be repeated several times a year as students leave the vocational 
courses in your school. 



The second phase of the program consists of a follow-up questionnaire 
which is sent from "Computer Central" to the former vocational students 
early in October of each year. In December, after the completed 
questionnaires are returned to Salt Lake City, the computer will analyze 
the information from the questionnaires in conformance to a specified 
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computer program. An Instructor's Record plus an analysis of the data 
obtained will be mailed to each participating school. A tentative copy 
of the student mail-out questionnaire is enclosed as Enclosure 3. 

We need your cooperation in this program not only for the consolidated 
state-wide data that will become available, but also because it will 
provide the individual schools with much of the data they will need for 
more sophisticated analyses of their own efforts in vocational education 
than is now available to them. These data will also serve as the basis 
for the Federal Report due in Washington D. C, each December. 

We will forward Exit Interview Forms (plus all necessary instructions) 
for your use beginning January 2nd. Prior to that date, we suggest 
you take two preparatory steps: 

First: Ask each of your students enrolled in a vocational class, grades 

10-12, to secure a social security number as soon as possible. This 
should be no particular hardship on the students since, as you know, 
they must have a social security number to get a driver's license, to 
start a bank account, and for many other purposes. But it is an 
important step because the computer will identify students and former 
students by their social security numbers to avoid difficulties caused 
by name changes, use of nicknames, etc. 

Second: At a faculty meeting or by other means of communication, explain 

the program to the vocational instructors and emphasize to them the 
importance of properly completed exit interview forms for each student 
leaving a class. The teachers, in turn, should be encouraged to emphasize 
to the students, soon after a course begins, the requirement for exit 
interviews and the reason for the follow-up, which is not an invasion 
of privacy, but an attempt to provide a better educational experience 
for their successors. 

This program has the wholehearted support of the State Superintendent 
of Public Instruction and also of your District Superintendent. With 
your help, the program should succeed. 

Thank you for your cooperation. 

JOHN F. STEPHENS, DIRECTOR 
UTAH RESEARCH COORDINATING UNIT 

By ... 

John A. Cox, Research Assistant 
JFS:JAC:mh 
Enclosures 3 
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DEAR FORMER STUDENT ^ pp * s '^° 1 * 

This is your follow-up questionnaire. Please respond 
and mail within ten da ys. 

PLEASE ANSWER ALL ENCLOSED CARDS 



$ 



V 



CIRCLE OR CHECK CORRECT ANSWER 



t Are you still in High School 


ye* 


no 


2. Did you graduate from H.S. 


yes 


no 


3. Did you leave H.S. before 
graduating 


yes 


no 


4. Are you on active duty in 
the military forces 


yes 


no 


5. Are you enrolled in post- 
secondary school 

full time 
part time 

a not at all 


£7 
£ 7 
£7 



NAME Of SCHOOL 



b. 



c. Level of program: 

Non- Baccalaureate CJ 

Baccalaureate EJ 

Graduate School CJ 

d. Type of post-sec. course: 

Same or related field as the 
voc. course indicated on the left 



yes no 

6. Did you seek employment in 
the occupation for which you 
trained in the above course 

yes no 

7 What is your present job title 

8. What is the name of the 
company for which you work 

9. Present name and home address 
(if different from name and 
address on the left): 
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APPENDIX C 



FOLLOW-UP QUESTIONNAIRE 
Circle or check correct answer 



Are you still attending high school full-time? 
Did you graduate from high school? 

Are you on active duty in the military forces? 
Are you presently employed? 



yes no 
yes jao 
yes no 

yes no (if no, go on to #5) 



a. 



If yes, are you employed: 

full-time 

part-time 



b. What is your present job title: 



c, 

d, 



What is the name of the company for which you work? 



What is your hourly wage? (check appropriate range) 

1. Under $1.00 4. $2.00-$2.50 

2. $1. 00«.$1.50 5. $2.50-$3.00 

3. $1.50-$2.00 6. over $3*00 



Would you like to work in your present occupational field for the next 5 years? 



1. 


definitely 


4. 


probably not 


2. 


probably 


5. 


definitely not 


3. 


uncertain 


6. 


other 



Explain 



a. 

b. 



If you are not employed , are you presently seeking work? yes 
If you are not seeking work, check the appropriate reason; 



no 



1 . 

2 . 

3. 



illness 

housewife 

in the Peace Corps, VISTA, etc. 



4. 

5. 



LDS Mission 
Other: 



Please check the name of any school or shcools in which you have been enrolled as a 
full-time student since you left high school. 



1 . 

“ 2 . 

‘3. 

"4. 



none 

University of Utah 
Brigham Young University 
Utah State University 



5. 

“6. 

7. 

“8. 



9. 

10 , 

11 , 



College of Eastern Utah 
Dixie College 
Snow College 
Weber State College 

Westminster College 

Stevens -Henager College 

LDS Business College 



12. Provo Trade Tech College 

13. Utah Technical College, Salt Lake 

14. another junior college, 

name: , 

15. another four-year college, 
name: 



16. another business or secretarial 
college: Name: 



17. another technical or trade school, 

name 

18. Other; name: 
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9. 



10. 



f 



r 



Are you currently attending one of the schools checked in question #6? 

_ ^ es ? I 3111 currently attending school (go on to question #8) 

Ne , I am not currently attending school (go on to question #ll) 



Please check the name of the school which 
(check one only) 

______ 1 . none 

2. University of Utah 

__ 3* Brigham Young University 

______ 4. Utah State University 

_____ 5* College of Eastern Utah 

_____ 6. Dixie College 

7. Snow College 

8. Weber State College 

_____ 9* Westminster College 

_____ 10. Stevens -Henager College 

_____ 11. LDS Business College 



What is your major or filed of 



are currently attending. 

12. Provo Trade Tech College 

~ 13. Utah Technical College, SLS 

_____ 14. another junior college, 
name : 

_____ 15 . another four-year college, 
name : 

16 . another business or secretarial 
college, name: 

17. another technical or trade school 

name : 

18. Other, Name: 



Please check the type of program in which you are enrolled. Check one answer only. 



1. ______ four-year college program 

Other than four-year program: 



j 2. general education 

3* vocational or technical education 

4. other (please specify) 

1 

'll. If the answer to question #7 is no, why did you leave school? 

I (Check the one most appropriate reason) 

graduated 

completed the course 
had a job offer 
to get married 
to look for a job 
not interested in the course 

to change schools (please specify which school you entered): 



I j 8. academic difficulty 

[ ( 9. other, (please specify) 

| | mmmmmmmrnmmmm 

?-\l2. Present name and home address, if different from the one at the top of questionnaire: 



1. 

2 . 

3. 

4. 

5. 

6. 



Name: 



Address : 



City, State, Zip: 
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APPENDIX II 



School 


Data Prom 15 -School Check Count 
Used to Establish Adjustment Factor 
Vocational Enrollment 
Reported in Spring '68 


Vocational Enrollment As 
Derived from Exit Forms 


Bingham 


456 


372 


Bonneville 


447 


307 


Bountiful 


690 


389 


‘Dugway 


82 


5 


East 


788 


529 


Grantsville 


165 


— 


Lehi 


238 


166 


Park City 


88 


23 


Provo 


843 


655 


South 


910 


171 


South Summit 


129 


75 


Tintic 


48 


48 


Wasatch 


282 


219 


Weber 


635 


595 


Wendover 


15 


9 




W£~ 


35S-T 



3563 

5816 

A ".6" 



= ,612 

correction factor was used. 



-26- 




rnmmmmMmmammm 






APPENDIX I 



Utah State Board of Education 



"Follow-up 1 * 



Criteria for Secondary-Level Program Completion 



1. Distributive Occupations (Marketing) 

Student has completed one of the following in the 10th, 11th, or 
12th grades: 

(1) A class in Salesmanship and Advertising which is a two-hour 
or longer block; or 

(2) A Cooperative Marketing course. 

2. Vocational Services 

Student has completed a class in the 10th, Ilth, or 12th grade in 
one of the following: 

(1) A "Vocational Services’* class; or 

(2) In the 11th or 12th grades has completed a two hour Vocational 
Service course. 

3. Health Occupations 

Student has completed in the 12th grade the course entitled: 

"Overview of Health Occupations". 

4. Office Occupations 

Student has completed any one or a combination of the following 
during the 10th, 11th, or 12th grades; 

(1) Typing - 3 or more semesters = 2 courses 

(2) Bookkeeping - 3 or more semesters - 2 courses 

(3) Any business data processing class 

(4) Shorthand - 3 or more semesters « 2 courses 

(5) Any filing class 

(6) Any full-year office practice class 

(7) Any full-year business machine class 

(8) Any cooperative business class 

(9) Any business simulation class 

(10) Any two or three hour block office occupation program (the 
block may be a combination of any of the above classes). 

5. Trade and Industrial Education 

Student has completed in the 11th or 12th grade one of the following: 

(1) Two or three hour block program; or 

(2) A cooperative T and I program. 
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6, Vocational Agriculture 

Student has completed tv/o or more years of vocational agriculture 
classes, 

7, Home Economics for Wage Earning (Gainful ) 

Student has completed in the 12t‘n grade a Home Economics course 
which meets the following criteria: 

(1) Job related (gainful) class; 

(2) Cooperative or simulation program; 

(3) Two hour or longer block class. 

8, Home Economics for Useful 

Student has completed in the 10th, 11th, or 12th grade one of 
the following: 

(1) A full year of any useful home economics class; or 

(2) A two-hour or longer block of any useful home economics 
class. 
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APPENDIX J 



Definitional Problems 



Status Unknown as used on the Federal Follow-up Report (Form OE 4045) 



and on local reports: 

Persons who returned a Mai lout Questionnaire but who failed to 



supply enough information to be assigned a status. Operationally 
this meant that they were not attending school full-time, were 
not employed full-time or part-time (in either the occupation for 
which trained, a related occupation, or unrelated occupation), 
were not in the military services, were not unemployed and seeking 



work, and had not voluntarily withdrawn from the labor force. 
Marketable skills as used in relation to dropouts on the Federal 



Follow-up Report and on Local Reports: 

Evidence of the student having acquired marketable skills in the 



occupation for which he was trained was a response on the Mailout 
Questionnaire that the student was working in the same or a related 



occupation, full-time or part-time. 
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Criteria for distinguishing TECHNICAL from TRADE AND IND IS TRIAL programs. 



NOTE: THERE ARE NO TECHNICAL COURSES TAUGHT IN THE HIGH SCHOOLS.* 



In TECHNICAL, 25% of the student's time or credit is devoted to science 
and mathematics courses which are not strictly occupationally-oriented, 

257* to general and related subjects, and 50% to theory and laboratory work. 
TRADE AND INDUSTRIAL courses devote more time to laboratory-shop work, 
thereby reducing the amount of mathematics, science, general and theoretical 
subjects. These are the standards adopted by the Vocational Division of 
the State Board of Education. Please, disregard whatever standards you 
usually employ in your individual schools in deciding whether to use the 
TECHNICAL code; 16.000 series, or the TRADE AND INDUSTRIAL codes, 17.000 
series. 

In addition to the foregoing criteria, the following may be of help co you 
in distinguishing TECHNICAL from TRADE AND INDUSTRIAL where it is a close 
question: 



TRADE AND INDUSTRIAL COURSE (17.000 series) 



TECHNICAL COURSE (16.000 series) 

1. Develops theorectical skills, 1. 

e.g., analysis of processes. 

2. May involve preparation of 2. 

technical reports and graphic 
analyses . 

3. Stresses the mathematical and 3. 
scientific bases of the occupa- 
tions. 

4. Requires at least a two year 4. 
training period of nine months 

per year, but does not include 
baccalaureate degree programs. 



Develops more practical skills, 
e.g., machine operation. 

Probably does not require advanced 
communication skills, yet deals 
with preparation for advancement 
to supervisory and foreman levels. 

Probably is more restricted to 
applied aspects of mathematics 
and science. 

May be less than a two year program. 



We are certain that the time and effort, not to mention expense, put into 
this project are worthwhile. Many other states have carried out a project 
similar to this one and have been highly successful. Thank you for your part 
in the Utah project. If there are any further questions or problems, 
let us know about them and we will be happy to help you. Call Mr. Bruce 
E. Humberstone, Research Assistant, 323-5891. 



